Graduate standards in law: outcomes

(Transcript of a section of the Graduate standards in law report, 1997)

Criteria for standards

Drawing on recent surveys and literature concerning law schools and confirmed by the meeting of experienced externals, the project considers that we can present some preliminary indications of the kind of approach which will correspond to opinion and practice within the sector and the areas in which further work will be needed.

As a starting point, it is worth noting that the mock moderating exercise confirmed the existence of a broad consensus among colleagues from different kinds of institution as to the marks to be given to candidates. Given the common curricular framework defined by the Law Society and Bar Council’s joint announcement on qualifying law degrees and by a broad similarity in the materials used to support student learning, such a convergence of approach is not surprising. This does provide a basis for developing common standards for the subject.

At the same time, it is recognised that there is wide diversity in the background and abilities of the student body and a diversity of mission among institutions. There was a widespread concern that a formal statement of standards would inhibit the continued flourishing of this diversity and would lead to a national curriculum. Given that recommendations of the Dearing Report to encourage wider access and greater diversity among the student body, the diversity of mission and student constituency among higher education institutions will properly continue. Any statement of standards must be sensitive to the diversity of institutions and students to whom it applies.

Domains

Our experience is that it is relatively uncontroversial to set out broad categories of achievement which a graduate should demonstrate in relation to subject knowledge and intellectual skills. On the whole, colleagues are prepared to accept something of this kind as a framework against which to map what they offer in their programmes.

They are, however, concerned that it becomes a constraining checklist which every programme of study must offer. There is also concern that the inevitable appearance of fragmentation which any list of domains produces should not work against a more holistic assessment of the achievements of a student.

In relation to other areas, there remain some concerns about particular aspects which are frequently included in the list of achievements graduates should demonstrate.

Professional responsibility

Following SEEAC we included professional responsibility in our draft list of domains. We have found this heading is not well understood and is considered by many to be inappropriate. Given that most institutions and ACLEC see a law degree as typically part of a liberal education, then training in professional standards is not part of the remit of the LLB. Even conceived in the broad way set out by SEEAC as encapsulating a professional approach to presentation and research, it is not recognised as a valuable distinct category. It does not form part of our final draft framework.

Key skills

There are three issues; relevance, integration, and recognition. There was concern expressed about the place of key skills within a subject-specific statement of graduate standards. It is recognised that many institutions do teach and assess achievements in personal and key skills as part of the law programme of study. The expectation is that students will be able to demonstrate the skills in the context of the subject.

But this integration approach is not universal. Often such skills are not specifically addressed in the teaching, feedback and assessment students receive. Particularly if numeracy is included among key skills, it may be more appropriate to state that students are expected to achieve a minimum standard in these areas on graduation, but that this may be the result of courses and activities laid on by the institution, not necessarily simply within the law modules studied.

It was recognised that achievement in personal and key skills may need some different form of recognition than the classification system. In the first place, this is because competence is normally recognised on a pass/fail basis. In the second place, performance in key skills may not be as easily related to levels as knowledge development. The area of key skills needs further work.

Typology

The typology produced did meet broad recognition. It is accepted that there is a proper difference in the achievement expected by examiners of students on academic and vocational programmes. Even within academic programmes, there are differences in what experienced externals expect from single-subject, mixed degree and subsidiary students. There is still work to be done to find appropriate ways of articulating this distinction of types, but it appears well established.

It is, however, problematic to give effect to such a distinction in the assessment process. Under modular arrangements, students from different programmes will often be taking the same module with a single set of objectives and a common mode of assessment. It may thus not be possible to differentiate between students doing particular law modules as part of different types of programme of study. If the standards achieved by these different cohorts of students should be recognised as distinct, then this would have implications for course design and the assessment process.

We have produced a revised version of these criteria on domains and typology and a graphical illustration of how these ideas could be expanded by a particular institution to establish outcome statements for a degree.

Levels

It would be right to recognise that there is a divergence of opinion among law teachers as to the appropriateness of using level descriptors. Many colleagues work in institutions which require them to assign modules to different levels and are comfortable with the idea that modules at level 2 are more difficult than those at level 1, and that there is an expectation that development of students is mapped by passing modules at higher levels.

By contrast, other colleagues argue that, especially in the core subjects required for professional purposes, the threshold standard is best described as a professional level of competence in the subject. The standard required is the same, whether the student studies a subject like contract at level 1, level 2 or level M.

Of courses, students at a higher level would be expected to achieve better marks, but the standard against which performance is measured is constant. Given that the prescribed subjects may appear in the early years of single subject law programmes, in the higher years of mixedsubject programmes, and in masters level courses for graduates in other subjects, the issue of levels is significant and has implications for the objectives of modules and their assessment.

Work on standards will also have to concern itself with the relationship to degrees of the Common Professional Examination (CPE) and Institute of Legal Executives (ILEX) qualifications. The CPE (for which a postgraduate diploma is often awarded) is a specific issue requiring further work. It is a conversion course designed for graduates in which the core qualifying subjects are studied.

ILEX examinations are often given some recognition in the form of advanced standing for degree programmes. But, at present, the professional bodies do not recognise HND courses in law as equivalent to any level of an undergraduate degree programme.

External examining

The experience of the workshop suggests that there is a valuable place for a regular meeting of external examiners to share experiences and to feed back to the sector suggestions for improvement. There were a significant number of common practical issues about which greater clarity and even commonality of practice might be desirable. A checklist of expectations on the procedure for communicating between externals and internals might be helpful in clarifying what is expected of each.

The meeting discussed the utility of various forms of criteria marking sheets and the background information which was considered helpful to enable the external to perform his job. Such comments on good practice ought to be disseminated more widely. The process of discussing issues was considered to be beneficial by many of the participants. While many would be resistant to the idea of training in external examining, principally on grounds of time, the idea of a meeting to discuss common issues from time to time was considered beneficial and would be usefully organised by subject associations.

There are a number of important questions concerning the present role of external examiners which will have to be clarified in any new system. In particular, the role of the external is unclear. Is the external supposed to certify that the various borderlines between classes have been drawn at appropriate points in individual cases or merely to certify, through sampling, that the pattern of grades awarded is appropriate? Is the external meant to ensure that the institution’s own standards are properly applied, or to base judgments on standards applied more widely in the sector? Is the external there to judge whether borderlines are rightly drawn by the internals (the role of the third marker) or only to intervene where the internal marks are seriously inappropriate?

The report suggests that the old processes by which internal examiners learnt the standards of their institution by osmosis are rapidly disappearing, so that formal processses of induction are needed for internals. The interactions of internal markers through sharing of exemplars and developing common understandings is an important part of the assessment process against which the external examiner system operates.

Current practice

The results of the questionnaire to law schools on assessment practices suggest that they typically use unseen written examinations for substantial part of their assessment processes, especially in relation to the core subjects, but that assessed coursework done on an individual basis by students in their own time is increasingly common.

On the whole, dissertations or other forms of extended research paper are merely optional features of undergraduate law programmes (unlike programmes in many other disciplines for whom a project is an essential feature of any demonstration of achieving graduate status). Although limited typically to optional subjects, group assessment and oral assessment are becoming common components of assessment.

The results here should be taken with the
1996 Harris survey of law schools, which identified a growth in the number of occasions on which assessment takes place in modular law courses. The resulting picture is that the number of components which fall to be moderated by an external examiner are increasing.

Group assessment and oral performance are often difficult for the external to moderate. The absence of a research dissertation in many courses means that the number of units to be assessed at the time of terminal examinations is higher than in many disciplines. The pattern of assessment and the consequences for the external which are peculiar to law need to be taken into account in the design of any new system for externals.

Current constraints

A questionnaire completed by the experienced externals present at the workshop gave some
broad indications of problems with the present system which any reform would have to address.

The two leading problem areas identified by colleagues were the time allowed by the examining institution for the external to provide comments and decisions on assessed coursework and scripts. In many institutions, the time allowed is very short and the number of items to be moderated is large.

The second area of concern was the pressure of work from the external’s own institution, which reduced time for performing the task effectively. The third major area of concern was that the external was insufficiently expert in the subject being moderated to pass comment. With the wide variety of options offered, it is very likely that no external has expertise in the relevant field so as to be able to judge whether the internal examiner has set an appropriate standard.

Concerns were also expressed about the Quality Assurance Agency’s proposals in the light of current experience of externals on degree level and LPC courses.

The way forward

We believe that any expert panel set up for law should seek to produce a range of acceptable frameworks within which a particular institution can benchmark its offerings. The purpose is clarity of information about courses, rather than constraining institutions into a homogeneous framework. It is important that the setting of graduate standards in law is brought into line with the joint announcement on qualifying law degrees. There should not be two distinct standards to which law departments have to work.

Stage 1

The panel should produce a framework around which consultation would take place:

• a list of domains of graduateness in which a student should demonstrate achievement
• a typology of purposes of different programmes

Stage 2

A set of consultations made up of regional workshops and invitations to comment. We think both are necessary to perform an educative, as well as a fact finding function. There should be genuine involvement of a wide range of teachers, who would be asked to produce their specification of threshold standards at levels 1, 3 and M in terms of outcomes expected for each. The issue of whether there should be a single level of competence in law could there be debated.

Some similar consultations among professionals might also be appropriate.

Stage 3

The expert panel would produce its statements having analysed responses and consult with representative organisations on the final formulation.

We think that stage 1 does not involve a lot of work, but stage 2 would require significant time, as would the collation of results in stage 3.

Subject associations have already provided much of the basis for stage 1. They would be able to organise regional workshops, but would need to be involved in planning. There would need to be at least a person month in collating responses and producing an appropriate range of syntheses for agreement by the expert panel.

Last Modified: 30 June 2010